{"nbformat":4,"nbformat_minor":0,"metadata":{"colab":{"name":"8_Multi_lingual_ner_pos_stop_words_sentiment_pretrained.ipynb","provenance":[],"collapsed_sections":[],"toc_visible":true},"kernelspec":{"name":"python3","display_name":"Python 3"}},"cells":[{"cell_type":"markdown","metadata":{"id":"83Xg78JEbhPB"},"source":["![JohnSnowLabs](https://nlp.johnsnowlabs.com/assets/images/logo.png)\n","\n","[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/JohnSnowLabs/nlu/blob/master/examples/webinars_conferences_etc/multi_lingual_webinar/8_Multi_lingual_ner_pos_stop_words_sentiment_pretrained.ipynb)\n"]},{"cell_type":"markdown","metadata":{"id":"JwvVcuGWblhd"},"source":["# Overview of some of the Multilingual modes avaiable in just 1 line of code with State Of the Art accuracy\n","\n","## Checkout [the Modelshub](https://nlp.johnsnowlabs.com/models) and the [NLU Namespace](https://nlu.johnsnowlabs.com/docs/en/spellbook) for more models and infos about them\n","![Flags](http://ckl-it.de/wp-content/uploads/2021/02/flags.jpeg)"]},{"cell_type":"code","metadata":{"id":"aa2FyC_LS-E0","colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"status":"ok","timestamp":1650025706021,"user_tz":-300,"elapsed":97661,"user":{"displayName":"ahmed lone","userId":"02458088882398909889"}},"outputId":"0a213b62-f702-4117-d61f-818735334417"},"source":["!wget https://setup.johnsnowlabs.com/nlu/colab.sh -O - | bash\n","import nlu"],"execution_count":1,"outputs":[{"output_type":"stream","name":"stdout","text":["--2022-04-15 12:26:48--  https://setup.johnsnowlabs.com/nlu/colab.sh\n","Resolving setup.johnsnowlabs.com (setup.johnsnowlabs.com)... 51.158.130.125\n","Connecting to setup.johnsnowlabs.com (setup.johnsnowlabs.com)|51.158.130.125|:443... connected.\n","HTTP request sent, awaiting response... 302 Moved Temporarily\n","Location: https://raw.githubusercontent.com/JohnSnowLabs/nlu/master/scripts/colab_setup.sh [following]\n","--2022-04-15 12:26:48--  https://raw.githubusercontent.com/JohnSnowLabs/nlu/master/scripts/colab_setup.sh\n","Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 185.199.108.133, 185.199.109.133, 185.199.110.133, ...\n","Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|185.199.108.133|:443... connected.\n","HTTP request sent, awaiting response... 200 OK\n","Length: 1665 (1.6K) [text/plain]\n","Saving to: ‘STDOUT’\n","\n","-                   100%[===================>]   1.63K  --.-KB/s    in 0s      \n","\n","2022-04-15 12:26:49 (34.3 MB/s) - written to stdout [1665/1665]\n","\n","Installing  NLU 3.4.3rc2 with  PySpark 3.0.3 and Spark NLP 3.4.2 for Google Colab ...\n","Ign:1 https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64  InRelease\n","Get:2 https://cloud.r-project.org/bin/linux/ubuntu bionic-cran40/ InRelease [3,626 B]\n","Ign:3 https://developer.download.nvidia.com/compute/machine-learning/repos/ubuntu1804/x86_64  InRelease\n","Get:4 https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64  Release [696 B]\n","Hit:5 https://developer.download.nvidia.com/compute/machine-learning/repos/ubuntu1804/x86_64  Release\n","Get:6 https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64  Release.gpg [836 B]\n","Hit:7 http://archive.ubuntu.com/ubuntu bionic InRelease\n","Get:8 http://archive.ubuntu.com/ubuntu bionic-updates InRelease [88.7 kB]\n","Get:9 http://ppa.launchpad.net/c2d4u.team/c2d4u4.0+/ubuntu bionic InRelease [15.9 kB]\n","Get:10 http://security.ubuntu.com/ubuntu bionic-security InRelease [88.7 kB]\n","Get:12 http://archive.ubuntu.com/ubuntu bionic-backports InRelease [74.6 kB]\n","Get:13 https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64  Packages [953 kB]\n","Hit:14 http://ppa.launchpad.net/cran/libgit2/ubuntu bionic InRelease\n","Get:15 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 Packages [2,268 kB]\n","Get:16 http://ppa.launchpad.net/deadsnakes/ppa/ubuntu bionic InRelease [15.9 kB]\n","Get:17 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 Packages [3,134 kB]\n","Hit:18 http://ppa.launchpad.net/graphics-drivers/ppa/ubuntu bionic InRelease\n","Get:19 http://ppa.launchpad.net/c2d4u.team/c2d4u4.0+/ubuntu bionic/main Sources [1,947 kB]\n","Get:20 http://security.ubuntu.com/ubuntu bionic-security/main amd64 Packages [2,695 kB]\n","Get:21 http://security.ubuntu.com/ubuntu bionic-security/universe amd64 Packages [1,490 kB]\n","Get:22 http://ppa.launchpad.net/c2d4u.team/c2d4u4.0+/ubuntu bionic/main amd64 Packages [996 kB]\n","Get:23 http://ppa.launchpad.net/deadsnakes/ppa/ubuntu bionic/main amd64 Packages [45.3 kB]\n","Fetched 13.8 MB in 6s (2,409 kB/s)\n","Reading package lists... Done\n","tar: spark-3.0.2-bin-hadoop2.7.tgz: Cannot open: No such file or directory\n","tar: Error is not recoverable: exiting now\n","\u001b[K     |████████████████████████████████| 209.1 MB 62 kB/s \n","\u001b[K     |████████████████████████████████| 142 kB 49.0 MB/s \n","\u001b[K     |████████████████████████████████| 505 kB 53.6 MB/s \n","\u001b[K     |████████████████████████████████| 198 kB 59.2 MB/s \n","\u001b[?25h  Building wheel for pyspark (setup.py) ... \u001b[?25l\u001b[?25hdone\n","Collecting nlu_tmp==3.4.3rc10\n","  Downloading nlu_tmp-3.4.3rc10-py3-none-any.whl (510 kB)\n","\u001b[K     |████████████████████████████████| 510 kB 7.9 MB/s \n","\u001b[?25hRequirement already satisfied: pandas>=1.3.5 in /usr/local/lib/python3.7/dist-packages (from nlu_tmp==3.4.3rc10) (1.3.5)\n","Requirement already satisfied: dataclasses in /usr/local/lib/python3.7/dist-packages (from nlu_tmp==3.4.3rc10) (0.6)\n","Requirement already satisfied: pyarrow>=0.16.0 in /usr/local/lib/python3.7/dist-packages (from nlu_tmp==3.4.3rc10) (6.0.1)\n","Requirement already satisfied: spark-nlp<3.5.0,>=3.4.2 in /usr/local/lib/python3.7/dist-packages (from nlu_tmp==3.4.3rc10) (3.4.2)\n","Requirement already satisfied: numpy in /usr/local/lib/python3.7/dist-packages (from nlu_tmp==3.4.3rc10) (1.21.5)\n","Requirement already satisfied: python-dateutil>=2.7.3 in /usr/local/lib/python3.7/dist-packages (from pandas>=1.3.5->nlu_tmp==3.4.3rc10) (2.8.2)\n","Requirement already satisfied: pytz>=2017.3 in /usr/local/lib/python3.7/dist-packages (from pandas>=1.3.5->nlu_tmp==3.4.3rc10) (2018.9)\n","Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.7/dist-packages (from python-dateutil>=2.7.3->pandas>=1.3.5->nlu_tmp==3.4.3rc10) (1.15.0)\n","Installing collected packages: nlu-tmp\n","Successfully installed nlu-tmp-3.4.3rc10\n"]}]},{"cell_type":"markdown","metadata":{"id":"Qyx1e-iWaIlF"},"source":["# [Bengali Lemmatizer](https://nlp.johnsnowlabs.com/2021/01/20/lemma_bn.html)\n","![bn](https://www.worldometers.info/img/flags/small/tn_bg-flag.gif)"]},{"cell_type":"code","metadata":{"colab":{"base_uri":"https://localhost:8080/","height":1000},"id":"HR8j_w3KZ0X-","executionInfo":{"status":"ok","timestamp":1650025754056,"user_tz":-300,"elapsed":48047,"user":{"displayName":"ahmed lone","userId":"02458088882398909889"}},"outputId":"a3e6f58e-4e80-4430-cf70-ad847dfce584"},"source":["#Bengali for :  'One morning in the marble-decorated building of Vaidyanatha, an obese monk was engaged in the enchantment of Duis and the milk service of one and a half Vaidyanatha. Give me two to eat'\n","nlu.load(\"bn.lemma\").predict(\"একদিন প্রাতে বৈদ্যনাথের মার্বলমণ্ডিত দালানে একটি স্থূলোদর সন্ন্যাসী দুইসের মোহনভোগ এবং দেড়সের দুগ্ধ সেবায় নিযুক্ত আছে বৈদ্যনাথ গায়ে একখানি চাদর দিয়া জোড়করে একান্ত বিনীতভাবে ভূতলে বসিয়া ভক্তিভরে পবিত্র ভোজনব্যাপার নিরীক্ষণ করিতেছিলেন এমন সময় কোনোমতে দ্বারীদের দৃষ্টি এড়াইয়া জীর্ণদেহ বালক সহিত একটি অতি শীর্ণকায়া রমণী গৃহে প্রবেশ করিয়া ক্ষীণস্বরে কহিল বাবু দুটি খেতে দাও\")"],"execution_count":2,"outputs":[{"output_type":"stream","name":"stdout","text":["lemma download started this may take some time.\n","Approximate size to download 90.6 KB\n","[OK!]\n","sentence_detector_dl download started this may take some time.\n","Approximate size to download 354.6 KB\n","[OK!]\n"]},{"output_type":"execute_result","data":{"text/plain":["            lem         token\n","0         একদিন         একদিন\n","0        প্রাতঃ        প্রাতে\n","0      বৈদ্যনাথ    বৈদ্যনাথের\n","0  মার্বলমণ্ডিত  মার্বলমণ্ডিত\n","0         দালান        দালানে\n","0            এক          একটি\n","0      স্থূলউদর      স্থূলোদর\n","0     সন্ন্যাসী     সন্ন্যাসী\n","0        দুইসের        দুইসের\n","0       মোহনভোগ       মোহনভোগ\n","0           এবং           এবং\n","0       দেড়সের       দেড়সের\n","0         দুগ্ধ         দুগ্ধ\n","0          সেবা         সেবায়\n","0       নিযুক্ত       নিযুক্ত\n","0           আছে           আছে\n","0      বৈদ্যনাথ      বৈদ্যনাথ\n","0            গা          গায়ে\n","0         একখান        একখানি\n","0          চাদর          চাদর\n","0         দেওয়া          দিয়া\n","0        জোড়কর       জোড়করে\n","0        একান্ত        একান্ত\n","0      বিনীতভাব     বিনীতভাবে\n","0          ভূতল         ভূতলে\n","0           বসা         বসিয়া\n","0      ভক্তিভরা      ভক্তিভরে\n","0        পবিত্র        পবিত্র\n","0   ভোজনব্যাপার   ভোজনব্যাপার\n","0      নিরীক্ষণ      নিরীক্ষণ\n","0           করা    করিতেছিলেন\n","0           এমন           এমন\n","0           সময়           সময়\n","0        কোনোমত       কোনোমতে\n","0        দ্বারী     দ্বারীদের\n","0        দৃষ্টি        দৃষ্টি\n","0        এড়ানো       এড়াইয়া\n","0      জীর্ণদেহ      জীর্ণদেহ\n","0          বালক          বালক\n","0          সহিত          সহিত\n","0            এক          একটি\n","0           অতি           অতি\n","0     শীর্ণকায়া     শীর্ণকায়া\n","0          রমণী          রমণী\n","0           গৃহ          গৃহে\n","0        প্রবেশ        প্রবেশ\n","0       বিশ্বাস         করিয়া\n","0     ক্ষীণস্বর    ক্ষীণস্বরে\n","0           কহা          কহিল\n","0          বাবু          বাবু\n","0           দুই          দুটি\n","0         খাওয়া          খেতে\n","0         দাওয়া           দাও"],"text/html":["\n","  <div id=\"df-eeab583c-0d80-4acd-9773-4981ec5b430f\">\n","    <div class=\"colab-df-container\">\n","      <div>\n","<style scoped>\n","    .dataframe tbody tr th:only-of-type {\n","        vertical-align: middle;\n","    }\n","\n","    .dataframe tbody tr th {\n","        vertical-align: top;\n","    }\n","\n","    .dataframe thead th {\n","        text-align: right;\n","    }\n","</style>\n","<table border=\"1\" class=\"dataframe\">\n","  <thead>\n","    <tr style=\"text-align: right;\">\n","      <th></th>\n","      <th>lem</th>\n","      <th>token</th>\n","    </tr>\n","  </thead>\n","  <tbody>\n","    <tr>\n","      <th>0</th>\n","      <td>একদিন</td>\n","      <td>একদিন</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>প্রাতঃ</td>\n","      <td>প্রাতে</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>বৈদ্যনাথ</td>\n","      <td>বৈদ্যনাথের</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>মার্বলমণ্ডিত</td>\n","      <td>মার্বলমণ্ডিত</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>দালান</td>\n","      <td>দালানে</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>এক</td>\n","      <td>একটি</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>স্থূলউদর</td>\n","      <td>স্থূলোদর</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>সন্ন্যাসী</td>\n","      <td>সন্ন্যাসী</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>দুইসের</td>\n","      <td>দুইসের</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>মোহনভোগ</td>\n","      <td>মোহনভোগ</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>এবং</td>\n","      <td>এবং</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>দেড়সের</td>\n","      <td>দেড়সের</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>দুগ্ধ</td>\n","      <td>দুগ্ধ</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>সেবা</td>\n","      <td>সেবায়</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>নিযুক্ত</td>\n","      <td>নিযুক্ত</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>আছে</td>\n","      <td>আছে</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>বৈদ্যনাথ</td>\n","      <td>বৈদ্যনাথ</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>গা</td>\n","      <td>গায়ে</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>একখান</td>\n","      <td>একখানি</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>চাদর</td>\n","      <td>চাদর</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>দেওয়া</td>\n","      <td>দিয়া</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>জোড়কর</td>\n","      <td>জোড়করে</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>একান্ত</td>\n","      <td>একান্ত</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>বিনীতভাব</td>\n","      <td>বিনীতভাবে</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>ভূতল</td>\n","      <td>ভূতলে</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>বসা</td>\n","      <td>বসিয়া</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>ভক্তিভরা</td>\n","      <td>ভক্তিভরে</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>পবিত্র</td>\n","      <td>পবিত্র</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>ভোজনব্যাপার</td>\n","      <td>ভোজনব্যাপার</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>নিরীক্ষণ</td>\n","      <td>নিরীক্ষণ</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>করা</td>\n","      <td>করিতেছিলেন</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>এমন</td>\n","      <td>এমন</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>সময়</td>\n","      <td>সময়</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>কোনোমত</td>\n","      <td>কোনোমতে</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>দ্বারী</td>\n","      <td>দ্বারীদের</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>দৃষ্টি</td>\n","      <td>দৃষ্টি</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>এড়ানো</td>\n","      <td>এড়াইয়া</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>জীর্ণদেহ</td>\n","      <td>জীর্ণদেহ</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>বালক</td>\n","      <td>বালক</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>সহিত</td>\n","      <td>সহিত</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>এক</td>\n","      <td>একটি</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>অতি</td>\n","      <td>অতি</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>শীর্ণকায়া</td>\n","      <td>শীর্ণকায়া</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>রমণী</td>\n","      <td>রমণী</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>গৃহ</td>\n","      <td>গৃহে</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>প্রবেশ</td>\n","      <td>প্রবেশ</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>বিশ্বাস</td>\n","      <td>করিয়া</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>ক্ষীণস্বর</td>\n","      <td>ক্ষীণস্বরে</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>কহা</td>\n","      <td>কহিল</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>বাবু</td>\n","      <td>বাবু</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>দুই</td>\n","      <td>দুটি</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>খাওয়া</td>\n","      <td>খেতে</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>দাওয়া</td>\n","      <td>দাও</td>\n","    </tr>\n","  </tbody>\n","</table>\n","</div>\n","      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-eeab583c-0d80-4acd-9773-4981ec5b430f')\"\n","              title=\"Convert this dataframe to an interactive table.\"\n","              style=\"display:none;\">\n","        \n","  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n","       width=\"24px\">\n","    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n","    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n","  </svg>\n","      </button>\n","      \n","  <style>\n","    .colab-df-container {\n","      display:flex;\n","      flex-wrap:wrap;\n","      gap: 12px;\n","    }\n","\n","    .colab-df-convert {\n","      background-color: #E8F0FE;\n","      border: none;\n","      border-radius: 50%;\n","      cursor: pointer;\n","      display: none;\n","      fill: #1967D2;\n","      height: 32px;\n","      padding: 0 0 0 0;\n","      width: 32px;\n","    }\n","\n","    .colab-df-convert:hover {\n","      background-color: #E2EBFA;\n","      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n","      fill: #174EA6;\n","    }\n","\n","    [theme=dark] .colab-df-convert {\n","      background-color: #3B4455;\n","      fill: #D2E3FC;\n","    }\n","\n","    [theme=dark] .colab-df-convert:hover {\n","      background-color: #434B5C;\n","      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n","      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n","      fill: #FFFFFF;\n","    }\n","  </style>\n","\n","      <script>\n","        const buttonEl =\n","          document.querySelector('#df-eeab583c-0d80-4acd-9773-4981ec5b430f button.colab-df-convert');\n","        buttonEl.style.display =\n","          google.colab.kernel.accessAllowed ? 'block' : 'none';\n","\n","        async function convertToInteractive(key) {\n","          const element = document.querySelector('#df-eeab583c-0d80-4acd-9773-4981ec5b430f');\n","          const dataTable =\n","            await google.colab.kernel.invokeFunction('convertToInteractive',\n","                                                     [key], {});\n","          if (!dataTable) return;\n","\n","          const docLinkHtml = 'Like what you see? Visit the ' +\n","            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n","            + ' to learn more about interactive tables.';\n","          element.innerHTML = '';\n","          dataTable['output_type'] = 'display_data';\n","          await google.colab.output.renderOutput(dataTable, element);\n","          const docLink = document.createElement('div');\n","          docLink.innerHTML = docLinkHtml;\n","          element.appendChild(docLink);\n","        }\n","      </script>\n","    </div>\n","  </div>\n","  "]},"metadata":{},"execution_count":2}]},{"cell_type":"markdown","metadata":{"id":"Um_h-L4AfTJ_"},"source":["![ja](https://www.worldometers.info/img/flags/small/tn_ja-flag.gif)\n","# [Japanese Lemmatizer](https://nlp.johnsnowlabs.com/2021/01/15/lemma_ja.html)"]},{"cell_type":"code","metadata":{"colab":{"base_uri":"https://localhost:8080/","height":246},"id":"528wDX5SfyBj","executionInfo":{"status":"ok","timestamp":1650025769327,"user_tz":-300,"elapsed":15287,"user":{"displayName":"ahmed lone","userId":"02458088882398909889"}},"outputId":"fcaac58b-f951-4628-d9a6-f804a617613c"},"source":["#Japanese for :  'Some residents were uncomfortable with this, but it seems that no one is now openly protesting or protesting.;\n","nlu.load(\"ja.lemma\").predict(\"これに不快感を示す住民はいましたが,現在,表立って反対や抗議の声を挙げている住民はいないようです。\")"],"execution_count":3,"outputs":[{"output_type":"stream","name":"stdout","text":["lemma download started this may take some time.\n","Approximate size to download 3.4 MB\n","[OK!]\n","sentence_detector_dl download started this may take some time.\n","Approximate size to download 354.6 KB\n","[OK!]\n"]},{"output_type":"execute_result","data":{"text/plain":["                                                 lem  \\\n","0  これに不快感を示す住民はいましたが,現在,表立って反対や抗議の声を挙げている住民はいないようです。   \n","\n","                                               token  \n","0  これに不快感を示す住民はいましたが,現在,表立って反対や抗議の声を挙げている住民はいないようです。  "],"text/html":["\n","  <div id=\"df-b8944954-131d-4628-b3f3-e5052bf16c2a\">\n","    <div class=\"colab-df-container\">\n","      <div>\n","<style scoped>\n","    .dataframe tbody tr th:only-of-type {\n","        vertical-align: middle;\n","    }\n","\n","    .dataframe tbody tr th {\n","        vertical-align: top;\n","    }\n","\n","    .dataframe thead th {\n","        text-align: right;\n","    }\n","</style>\n","<table border=\"1\" class=\"dataframe\">\n","  <thead>\n","    <tr style=\"text-align: right;\">\n","      <th></th>\n","      <th>lem</th>\n","      <th>token</th>\n","    </tr>\n","  </thead>\n","  <tbody>\n","    <tr>\n","      <th>0</th>\n","      <td>これに不快感を示す住民はいましたが,現在,表立って反対や抗議の声を挙げている住民はいないようです。</td>\n","      <td>これに不快感を示す住民はいましたが,現在,表立って反対や抗議の声を挙げている住民はいないようです。</td>\n","    </tr>\n","  </tbody>\n","</table>\n","</div>\n","      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-b8944954-131d-4628-b3f3-e5052bf16c2a')\"\n","              title=\"Convert this dataframe to an interactive table.\"\n","              style=\"display:none;\">\n","        \n","  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n","       width=\"24px\">\n","    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n","    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n","  </svg>\n","      </button>\n","      \n","  <style>\n","    .colab-df-container {\n","      display:flex;\n","      flex-wrap:wrap;\n","      gap: 12px;\n","    }\n","\n","    .colab-df-convert {\n","      background-color: #E8F0FE;\n","      border: none;\n","      border-radius: 50%;\n","      cursor: pointer;\n","      display: none;\n","      fill: #1967D2;\n","      height: 32px;\n","      padding: 0 0 0 0;\n","      width: 32px;\n","    }\n","\n","    .colab-df-convert:hover {\n","      background-color: #E2EBFA;\n","      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n","      fill: #174EA6;\n","    }\n","\n","    [theme=dark] .colab-df-convert {\n","      background-color: #3B4455;\n","      fill: #D2E3FC;\n","    }\n","\n","    [theme=dark] .colab-df-convert:hover {\n","      background-color: #434B5C;\n","      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n","      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n","      fill: #FFFFFF;\n","    }\n","  </style>\n","\n","      <script>\n","        const buttonEl =\n","          document.querySelector('#df-b8944954-131d-4628-b3f3-e5052bf16c2a button.colab-df-convert');\n","        buttonEl.style.display =\n","          google.colab.kernel.accessAllowed ? 'block' : 'none';\n","\n","        async function convertToInteractive(key) {\n","          const element = document.querySelector('#df-b8944954-131d-4628-b3f3-e5052bf16c2a');\n","          const dataTable =\n","            await google.colab.kernel.invokeFunction('convertToInteractive',\n","                                                     [key], {});\n","          if (!dataTable) return;\n","\n","          const docLinkHtml = 'Like what you see? Visit the ' +\n","            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n","            + ' to learn more about interactive tables.';\n","          element.innerHTML = '';\n","          dataTable['output_type'] = 'display_data';\n","          await google.colab.output.renderOutput(dataTable, element);\n","          const docLink = document.createElement('div');\n","          docLink.innerHTML = docLinkHtml;\n","          element.appendChild(docLink);\n","        }\n","      </script>\n","    </div>\n","  </div>\n","  "]},"metadata":{},"execution_count":3}]},{"cell_type":"markdown","metadata":{"id":"n6Qm7iIpgwVj"},"source":["# [Amharic Lemmatizer](https://nlp.johnsnowlabs.com/2021/01/20/lemma_am.html)\n","![am](https://www.worldometers.info/img/flags/small/tn_am-flag.gif)\n"]},{"cell_type":"code","metadata":{"colab":{"base_uri":"https://localhost:8080/","height":436},"id":"tfIApxfigJww","executionInfo":{"status":"ok","timestamp":1650025779528,"user_tz":-300,"elapsed":10223,"user":{"displayName":"ahmed lone","userId":"02458088882398909889"}},"outputId":"de793f6f-2b59-41e9-a4bc-ee483a594d3e"},"source":["#Aharic for :  'Bookmark the permalink.'\n","nlu.load(\"am.lemma\").predict(\"መጽሐፉን መጽሐፍ ኡ ን አስያዛት አስያዝ ኧ ኣት ።\")"],"execution_count":4,"outputs":[{"output_type":"stream","name":"stdout","text":["lemma download started this may take some time.\n","Approximate size to download 36 KB\n","[OK!]\n","sentence_detector_dl download started this may take some time.\n","Approximate size to download 354.6 KB\n","[OK!]\n"]},{"output_type":"execute_result","data":{"text/plain":["    lem  token\n","0     _  መጽሐፉን\n","0  መጽሐፍ   መጽሐፍ\n","0     ኡ      ኡ\n","0     ን      ን\n","0     _  አስያዛት\n","0  አስያዝ   አስያዝ\n","0     ኧ      ኧ\n","0    ኣት     ኣት\n","0     ።      ።"],"text/html":["\n","  <div id=\"df-3096f2a0-c867-4b17-afc8-6e8ec4da254b\">\n","    <div class=\"colab-df-container\">\n","      <div>\n","<style scoped>\n","    .dataframe tbody tr th:only-of-type {\n","        vertical-align: middle;\n","    }\n","\n","    .dataframe tbody tr th {\n","        vertical-align: top;\n","    }\n","\n","    .dataframe thead th {\n","        text-align: right;\n","    }\n","</style>\n","<table border=\"1\" class=\"dataframe\">\n","  <thead>\n","    <tr style=\"text-align: right;\">\n","      <th></th>\n","      <th>lem</th>\n","      <th>token</th>\n","    </tr>\n","  </thead>\n","  <tbody>\n","    <tr>\n","      <th>0</th>\n","      <td>_</td>\n","      <td>መጽሐፉን</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>መጽሐፍ</td>\n","      <td>መጽሐፍ</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>ኡ</td>\n","      <td>ኡ</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>ን</td>\n","      <td>ን</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>_</td>\n","      <td>አስያዛት</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>አስያዝ</td>\n","      <td>አስያዝ</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>ኧ</td>\n","      <td>ኧ</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>ኣት</td>\n","      <td>ኣት</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>።</td>\n","      <td>።</td>\n","    </tr>\n","  </tbody>\n","</table>\n","</div>\n","      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-3096f2a0-c867-4b17-afc8-6e8ec4da254b')\"\n","              title=\"Convert this dataframe to an interactive table.\"\n","              style=\"display:none;\">\n","        \n","  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n","       width=\"24px\">\n","    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n","    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n","  </svg>\n","      </button>\n","      \n","  <style>\n","    .colab-df-container {\n","      display:flex;\n","      flex-wrap:wrap;\n","      gap: 12px;\n","    }\n","\n","    .colab-df-convert {\n","      background-color: #E8F0FE;\n","      border: none;\n","      border-radius: 50%;\n","      cursor: pointer;\n","      display: none;\n","      fill: #1967D2;\n","      height: 32px;\n","      padding: 0 0 0 0;\n","      width: 32px;\n","    }\n","\n","    .colab-df-convert:hover {\n","      background-color: #E2EBFA;\n","      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n","      fill: #174EA6;\n","    }\n","\n","    [theme=dark] .colab-df-convert {\n","      background-color: #3B4455;\n","      fill: #D2E3FC;\n","    }\n","\n","    [theme=dark] .colab-df-convert:hover {\n","      background-color: #434B5C;\n","      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n","      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n","      fill: #FFFFFF;\n","    }\n","  </style>\n","\n","      <script>\n","        const buttonEl =\n","          document.querySelector('#df-3096f2a0-c867-4b17-afc8-6e8ec4da254b button.colab-df-convert');\n","        buttonEl.style.display =\n","          google.colab.kernel.accessAllowed ? 'block' : 'none';\n","\n","        async function convertToInteractive(key) {\n","          const element = document.querySelector('#df-3096f2a0-c867-4b17-afc8-6e8ec4da254b');\n","          const dataTable =\n","            await google.colab.kernel.invokeFunction('convertToInteractive',\n","                                                     [key], {});\n","          if (!dataTable) return;\n","\n","          const docLinkHtml = 'Like what you see? Visit the ' +\n","            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n","            + ' to learn more about interactive tables.';\n","          element.innerHTML = '';\n","          dataTable['output_type'] = 'display_data';\n","          await google.colab.output.renderOutput(dataTable, element);\n","          const docLink = document.createElement('div');\n","          docLink.innerHTML = docLinkHtml;\n","          element.appendChild(docLink);\n","        }\n","      </script>\n","    </div>\n","  </div>\n","  "]},"metadata":{},"execution_count":4}]},{"cell_type":"markdown","metadata":{"id":"-hO6rGcFhUq8"},"source":["# [Bhojpuri Lemmatizer](https://nlp.johnsnowlabs.com/2021/01/18/lemma_bh.html)\n","![bh](https://www.worldometers.info/img/flags/small/tn_bh-flag.gif)"]},{"cell_type":"code","metadata":{"colab":{"base_uri":"https://localhost:8080/","height":1000},"id":"4cTM0dKGgQC1","executionInfo":{"status":"ok","timestamp":1650025785370,"user_tz":-300,"elapsed":5863,"user":{"displayName":"ahmed lone","userId":"02458088882398909889"}},"outputId":"5a895103-f1f0-429f-82ae-c049776a6eec"},"source":["#Bhojpuri for : 'In this event, participation of World Bhojpuri Conference, Purvanchal Ekta Manch, Veer Kunwar Singh Foundation, Purvanchal Bhojpuri Mahasabha, and Herf - Media.'\n","nlu.load(\"bh.lemma\").predict(\"एह आयोजन में विश्व भोजपुरी सम्मेलन , पूर्वांचल एकता मंच , वीर कुँवर सिंह फाउन्डेशन , पूर्वांचल भोजपुरी महासभा , अउर हर्फ - मीडिया के सहभागिता बा ।\")"],"execution_count":5,"outputs":[{"output_type":"stream","name":"stdout","text":["lemma download started this may take some time.\n","Approximate size to download 24.3 KB\n","[OK!]\n","sentence_detector_dl download started this may take some time.\n","Approximate size to download 354.6 KB\n","[OK!]\n"]},{"output_type":"execute_result","data":{"text/plain":["         lem      token\n","0         एह         एह\n","0      आयोजन      आयोजन\n","0        में        में\n","0      विश्व      विश्व\n","0    भोजपुरी    भोजपुरी\n","0    सम्मेलन    सम्मेलन\n","0      COMMA          ,\n","0  पूर्वांचल  पूर्वांचल\n","0       एकता       एकता\n","0        मंच        मंच\n","0      COMMA          ,\n","0        वीर        वीर\n","0      कुँवर      कुँवर\n","0       सिंह       सिंह\n","0  फाउन्डेशन  फाउन्डेशन\n","0      COMMA          ,\n","0  पूर्वांचल  पूर्वांचल\n","0    भोजपुरी    भोजपुरी\n","0     महासभा     महासभा\n","0      COMMA          ,\n","0        अउर        अउर\n","0       हर्फ       हर्फ\n","0          -          -\n","0     मीडिया     मीडिया\n","0         को         के\n","0   सहभागिता   सहभागिता\n","0         बा         बा\n","0          ।          ।"],"text/html":["\n","  <div id=\"df-e5ce458b-7614-495c-838f-5aed89c9dca9\">\n","    <div class=\"colab-df-container\">\n","      <div>\n","<style scoped>\n","    .dataframe tbody tr th:only-of-type {\n","        vertical-align: middle;\n","    }\n","\n","    .dataframe tbody tr th {\n","        vertical-align: top;\n","    }\n","\n","    .dataframe thead th {\n","        text-align: right;\n","    }\n","</style>\n","<table border=\"1\" class=\"dataframe\">\n","  <thead>\n","    <tr style=\"text-align: right;\">\n","      <th></th>\n","      <th>lem</th>\n","      <th>token</th>\n","    </tr>\n","  </thead>\n","  <tbody>\n","    <tr>\n","      <th>0</th>\n","      <td>एह</td>\n","      <td>एह</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>आयोजन</td>\n","      <td>आयोजन</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>में</td>\n","      <td>में</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>विश्व</td>\n","      <td>विश्व</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>भोजपुरी</td>\n","      <td>भोजपुरी</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>सम्मेलन</td>\n","      <td>सम्मेलन</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>COMMA</td>\n","      <td>,</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>पूर्वांचल</td>\n","      <td>पूर्वांचल</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>एकता</td>\n","      <td>एकता</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>मंच</td>\n","      <td>मंच</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>COMMA</td>\n","      <td>,</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>वीर</td>\n","      <td>वीर</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>कुँवर</td>\n","      <td>कुँवर</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>सिंह</td>\n","      <td>सिंह</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>फाउन्डेशन</td>\n","      <td>फाउन्डेशन</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>COMMA</td>\n","      <td>,</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>पूर्वांचल</td>\n","      <td>पूर्वांचल</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>भोजपुरी</td>\n","      <td>भोजपुरी</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>महासभा</td>\n","      <td>महासभा</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>COMMA</td>\n","      <td>,</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>अउर</td>\n","      <td>अउर</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>हर्फ</td>\n","      <td>हर्फ</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>-</td>\n","      <td>-</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>मीडिया</td>\n","      <td>मीडिया</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>को</td>\n","      <td>के</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>सहभागिता</td>\n","      <td>सहभागिता</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>बा</td>\n","      <td>बा</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>।</td>\n","      <td>।</td>\n","    </tr>\n","  </tbody>\n","</table>\n","</div>\n","      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-e5ce458b-7614-495c-838f-5aed89c9dca9')\"\n","              title=\"Convert this dataframe to an interactive table.\"\n","              style=\"display:none;\">\n","        \n","  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n","       width=\"24px\">\n","    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n","    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n","  </svg>\n","      </button>\n","      \n","  <style>\n","    .colab-df-container {\n","      display:flex;\n","      flex-wrap:wrap;\n","      gap: 12px;\n","    }\n","\n","    .colab-df-convert {\n","      background-color: #E8F0FE;\n","      border: none;\n","      border-radius: 50%;\n","      cursor: pointer;\n","      display: none;\n","      fill: #1967D2;\n","      height: 32px;\n","      padding: 0 0 0 0;\n","      width: 32px;\n","    }\n","\n","    .colab-df-convert:hover {\n","      background-color: #E2EBFA;\n","      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n","      fill: #174EA6;\n","    }\n","\n","    [theme=dark] .colab-df-convert {\n","      background-color: #3B4455;\n","      fill: #D2E3FC;\n","    }\n","\n","    [theme=dark] .colab-df-convert:hover {\n","      background-color: #434B5C;\n","      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n","      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n","      fill: #FFFFFF;\n","    }\n","  </style>\n","\n","      <script>\n","        const buttonEl =\n","          document.querySelector('#df-e5ce458b-7614-495c-838f-5aed89c9dca9 button.colab-df-convert');\n","        buttonEl.style.display =\n","          google.colab.kernel.accessAllowed ? 'block' : 'none';\n","\n","        async function convertToInteractive(key) {\n","          const element = document.querySelector('#df-e5ce458b-7614-495c-838f-5aed89c9dca9');\n","          const dataTable =\n","            await google.colab.kernel.invokeFunction('convertToInteractive',\n","                                                     [key], {});\n","          if (!dataTable) return;\n","\n","          const docLinkHtml = 'Like what you see? Visit the ' +\n","            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n","            + ' to learn more about interactive tables.';\n","          element.innerHTML = '';\n","          dataTable['output_type'] = 'display_data';\n","          await google.colab.output.renderOutput(dataTable, element);\n","          const docLink = document.createElement('div');\n","          docLink.innerHTML = docLinkHtml;\n","          element.appendChild(docLink);\n","        }\n","      </script>\n","    </div>\n","  </div>\n","  "]},"metadata":{},"execution_count":5}]},{"cell_type":"markdown","metadata":{"id":"jM6qeXmaWEH5"},"source":["# [Traditional Chinese Word Segmentation](https://nlp.johnsnowlabs.com/2021/01/25/wordseg_gsd_ud_trad_zh.html)\n","![zh](https://www.worldometers.info/img/flags/small/tn_ch-flag.gif)"]},{"cell_type":"code","metadata":{"colab":{"base_uri":"https://localhost:8080/","height":498},"id":"8aJ-cqLqWDWH","executionInfo":{"status":"ok","timestamp":1650025794053,"user_tz":-300,"elapsed":8705,"user":{"displayName":"ahmed lone","userId":"02458088882398909889"}},"outputId":"b0fd5137-5a6a-4ce1-f7aa-ddac01b7b1f9"},"source":["#  Chinese for :'However, this treatment also creates some problems' \n","nlu.load(\"zh.segment_words\").predict(\"然而，這樣的處理也衍生了一些問題。\")"],"execution_count":6,"outputs":[{"output_type":"stream","name":"stdout","text":["wordseg_weibo download started this may take some time.\n","Approximate size to download 1.1 MB\n","[OK!]\n","sentence_detector_dl download started this may take some time.\n","Approximate size to download 354.6 KB\n","[OK!]\n"]},{"output_type":"execute_result","data":{"text/plain":["  words_seg\n","0        然而\n","0         ，\n","0        這樣\n","0         的\n","0        處理\n","0         也\n","0        衍生\n","0         了\n","0        一些\n","0        問題\n","0         。"],"text/html":["\n","  <div id=\"df-41492530-dae7-4871-9b37-85db26e37b78\">\n","    <div class=\"colab-df-container\">\n","      <div>\n","<style scoped>\n","    .dataframe tbody tr th:only-of-type {\n","        vertical-align: middle;\n","    }\n","\n","    .dataframe tbody tr th {\n","        vertical-align: top;\n","    }\n","\n","    .dataframe thead th {\n","        text-align: right;\n","    }\n","</style>\n","<table border=\"1\" class=\"dataframe\">\n","  <thead>\n","    <tr style=\"text-align: right;\">\n","      <th></th>\n","      <th>words_seg</th>\n","    </tr>\n","  </thead>\n","  <tbody>\n","    <tr>\n","      <th>0</th>\n","      <td>然而</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>，</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>這樣</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>的</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>處理</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>也</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>衍生</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>了</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>一些</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>問題</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>。</td>\n","    </tr>\n","  </tbody>\n","</table>\n","</div>\n","      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-41492530-dae7-4871-9b37-85db26e37b78')\"\n","              title=\"Convert this dataframe to an interactive table.\"\n","              style=\"display:none;\">\n","        \n","  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n","       width=\"24px\">\n","    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n","    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n","  </svg>\n","      </button>\n","      \n","  <style>\n","    .colab-df-container {\n","      display:flex;\n","      flex-wrap:wrap;\n","      gap: 12px;\n","    }\n","\n","    .colab-df-convert {\n","      background-color: #E8F0FE;\n","      border: none;\n","      border-radius: 50%;\n","      cursor: pointer;\n","      display: none;\n","      fill: #1967D2;\n","      height: 32px;\n","      padding: 0 0 0 0;\n","      width: 32px;\n","    }\n","\n","    .colab-df-convert:hover {\n","      background-color: #E2EBFA;\n","      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n","      fill: #174EA6;\n","    }\n","\n","    [theme=dark] .colab-df-convert {\n","      background-color: #3B4455;\n","      fill: #D2E3FC;\n","    }\n","\n","    [theme=dark] .colab-df-convert:hover {\n","      background-color: #434B5C;\n","      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n","      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n","      fill: #FFFFFF;\n","    }\n","  </style>\n","\n","      <script>\n","        const buttonEl =\n","          document.querySelector('#df-41492530-dae7-4871-9b37-85db26e37b78 button.colab-df-convert');\n","        buttonEl.style.display =\n","          google.colab.kernel.accessAllowed ? 'block' : 'none';\n","\n","        async function convertToInteractive(key) {\n","          const element = document.querySelector('#df-41492530-dae7-4871-9b37-85db26e37b78');\n","          const dataTable =\n","            await google.colab.kernel.invokeFunction('convertToInteractive',\n","                                                     [key], {});\n","          if (!dataTable) return;\n","\n","          const docLinkHtml = 'Like what you see? Visit the ' +\n","            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n","            + ' to learn more about interactive tables.';\n","          element.innerHTML = '';\n","          dataTable['output_type'] = 'display_data';\n","          await google.colab.output.renderOutput(dataTable, element);\n","          const docLink = document.createElement('div');\n","          docLink.innerHTML = docLinkHtml;\n","          element.appendChild(docLink);\n","        }\n","      </script>\n","    </div>\n","  </div>\n","  "]},"metadata":{},"execution_count":6}]},{"cell_type":"markdown","metadata":{"id":"Om21rc2EWgSd"},"source":["# [Part of Speech for Traditional Chinese](https://nlp.johnsnowlabs.com/2021/01/25/pos_ud_gsd_trad_zh.html)\n","![zh](https://www.worldometers.info/img/flags/small/tn_ch-flag.gif)"]},{"cell_type":"code","metadata":{"colab":{"base_uri":"https://localhost:8080/","height":185},"id":"UuRkWQNHWHne","executionInfo":{"status":"ok","timestamp":1650025807290,"user_tz":-300,"elapsed":13255,"user":{"displayName":"ahmed lone","userId":"02458088882398909889"}},"outputId":"cb80b584-016c-410b-d6a7-e1e01e488eb4"},"source":["# ' Chinese for : 'However, this treatment also creates some problems' \n","nlu.load(\"zh.pos\").predict(\"然而，這樣的處理也衍生了一些問題。\")"],"execution_count":7,"outputs":[{"output_type":"stream","name":"stdout","text":["pos_ud_gsd download started this may take some time.\n","Approximate size to download 3.3 MB\n","[OK!]\n","sentence_detector_dl download started this may take some time.\n","Approximate size to download 354.6 KB\n","[OK!]\n"]},{"output_type":"execute_result","data":{"text/plain":["     pos              token\n","0  PUNCT  然而，這樣的處理也衍生了一些問題。"],"text/html":["\n","  <div id=\"df-40628087-a3da-4667-9561-9e52387e68d6\">\n","    <div class=\"colab-df-container\">\n","      <div>\n","<style scoped>\n","    .dataframe tbody tr th:only-of-type {\n","        vertical-align: middle;\n","    }\n","\n","    .dataframe tbody tr th {\n","        vertical-align: top;\n","    }\n","\n","    .dataframe thead th {\n","        text-align: right;\n","    }\n","</style>\n","<table border=\"1\" class=\"dataframe\">\n","  <thead>\n","    <tr style=\"text-align: right;\">\n","      <th></th>\n","      <th>pos</th>\n","      <th>token</th>\n","    </tr>\n","  </thead>\n","  <tbody>\n","    <tr>\n","      <th>0</th>\n","      <td>PUNCT</td>\n","      <td>然而，這樣的處理也衍生了一些問題。</td>\n","    </tr>\n","  </tbody>\n","</table>\n","</div>\n","      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-40628087-a3da-4667-9561-9e52387e68d6')\"\n","              title=\"Convert this dataframe to an interactive table.\"\n","              style=\"display:none;\">\n","        \n","  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n","       width=\"24px\">\n","    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n","    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n","  </svg>\n","      </button>\n","      \n","  <style>\n","    .colab-df-container {\n","      display:flex;\n","      flex-wrap:wrap;\n","      gap: 12px;\n","    }\n","\n","    .colab-df-convert {\n","      background-color: #E8F0FE;\n","      border: none;\n","      border-radius: 50%;\n","      cursor: pointer;\n","      display: none;\n","      fill: #1967D2;\n","      height: 32px;\n","      padding: 0 0 0 0;\n","      width: 32px;\n","    }\n","\n","    .colab-df-convert:hover {\n","      background-color: #E2EBFA;\n","      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n","      fill: #174EA6;\n","    }\n","\n","    [theme=dark] .colab-df-convert {\n","      background-color: #3B4455;\n","      fill: #D2E3FC;\n","    }\n","\n","    [theme=dark] .colab-df-convert:hover {\n","      background-color: #434B5C;\n","      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n","      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n","      fill: #FFFFFF;\n","    }\n","  </style>\n","\n","      <script>\n","        const buttonEl =\n","          document.querySelector('#df-40628087-a3da-4667-9561-9e52387e68d6 button.colab-df-convert');\n","        buttonEl.style.display =\n","          google.colab.kernel.accessAllowed ? 'block' : 'none';\n","\n","        async function convertToInteractive(key) {\n","          const element = document.querySelector('#df-40628087-a3da-4667-9561-9e52387e68d6');\n","          const dataTable =\n","            await google.colab.kernel.invokeFunction('convertToInteractive',\n","                                                     [key], {});\n","          if (!dataTable) return;\n","\n","          const docLinkHtml = 'Like what you see? Visit the ' +\n","            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n","            + ' to learn more about interactive tables.';\n","          element.innerHTML = '';\n","          dataTable['output_type'] = 'display_data';\n","          await google.colab.output.renderOutput(dataTable, element);\n","          const docLink = document.createElement('div');\n","          docLink.innerHTML = docLinkHtml;\n","          element.appendChild(docLink);\n","        }\n","      </script>\n","    </div>\n","  </div>\n","  "]},"metadata":{},"execution_count":7}]},{"cell_type":"markdown","metadata":{"id":"8u-11tBOXCfC"},"source":["# [Thai Word Segment Recognition](https://nlp.johnsnowlabs.com/2021/01/11/ner_lst20_glove_840B_300d_th.html)\n","![th](https://www.worldometers.info/img/flags/small/tn_th-flag.gif)\n"]},{"cell_type":"code","metadata":{"colab":{"base_uri":"https://localhost:8080/","height":1000},"id":"FAVVGBU8Wj5q","executionInfo":{"status":"ok","timestamp":1650025814783,"user_tz":-300,"elapsed":7562,"user":{"displayName":"ahmed lone","userId":"02458088882398909889"}},"outputId":"683cbcc7-a7c7-4016-9a3f-6cbc0e33c318"},"source":["# Thai for :  'Mona Lisa is a 16th-century oil painting created by Leonardo held at the Louvre in Paris' \n","nlu.load(\"th.segment_words\").predict(\"Mona Lisa เป็นภาพวาดสีน้ำมันในศตวรรษที่ 16 ที่สร้างโดย Leonardo จัดขึ้นที่พิพิธภัณฑ์ลูฟร์ในปารีส\")"],"execution_count":8,"outputs":[{"output_type":"stream","name":"stdout","text":["wordseg_best download started this may take some time.\n","Approximate size to download 79.2 KB\n","[OK!]\n","sentence_detector_dl download started this may take some time.\n","Approximate size to download 354.6 KB\n","[OK!]\n"]},{"output_type":"execute_result","data":{"text/plain":["    words_seg\n","0    MonaLisa\n","0        เป็น\n","0         ภาพ\n","0           ว\n","0           า\n","0           ด\n","0       สีน้ำ\n","0         มัน\n","0          ใน\n","0      ศตวรรษ\n","0         ที่\n","0          16\n","0         ที่\n","0       สร้าง\n","0           โ\n","0           ด\n","0           ย\n","0           L\n","0           e\n","0           o\n","0           n\n","0           a\n","0           r\n","0           d\n","0           o\n","0         จัด\n","0        ขึ้น\n","0         ที่\n","0  พิพิธภัณฑ์\n","0       ลูฟร์\n","0          ใน\n","0       ปารีส"],"text/html":["\n","  <div id=\"df-5b6b8314-4eb4-4c3f-b8d2-e91e119e63da\">\n","    <div class=\"colab-df-container\">\n","      <div>\n","<style scoped>\n","    .dataframe tbody tr th:only-of-type {\n","        vertical-align: middle;\n","    }\n","\n","    .dataframe tbody tr th {\n","        vertical-align: top;\n","    }\n","\n","    .dataframe thead th {\n","        text-align: right;\n","    }\n","</style>\n","<table border=\"1\" class=\"dataframe\">\n","  <thead>\n","    <tr style=\"text-align: right;\">\n","      <th></th>\n","      <th>words_seg</th>\n","    </tr>\n","  </thead>\n","  <tbody>\n","    <tr>\n","      <th>0</th>\n","      <td>MonaLisa</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>เป็น</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>ภาพ</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>ว</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>า</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>ด</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>สีน้ำ</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>มัน</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>ใน</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>ศตวรรษ</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>ที่</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>16</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>ที่</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>สร้าง</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>โ</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>ด</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>ย</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>L</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>e</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>o</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>n</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>a</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>r</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>d</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>o</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>จัด</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>ขึ้น</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>ที่</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>พิพิธภัณฑ์</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>ลูฟร์</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>ใน</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>ปารีส</td>\n","    </tr>\n","  </tbody>\n","</table>\n","</div>\n","      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-5b6b8314-4eb4-4c3f-b8d2-e91e119e63da')\"\n","              title=\"Convert this dataframe to an interactive table.\"\n","              style=\"display:none;\">\n","        \n","  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n","       width=\"24px\">\n","    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n","    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n","  </svg>\n","      </button>\n","      \n","  <style>\n","    .colab-df-container {\n","      display:flex;\n","      flex-wrap:wrap;\n","      gap: 12px;\n","    }\n","\n","    .colab-df-convert {\n","      background-color: #E8F0FE;\n","      border: none;\n","      border-radius: 50%;\n","      cursor: pointer;\n","      display: none;\n","      fill: #1967D2;\n","      height: 32px;\n","      padding: 0 0 0 0;\n","      width: 32px;\n","    }\n","\n","    .colab-df-convert:hover {\n","      background-color: #E2EBFA;\n","      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n","      fill: #174EA6;\n","    }\n","\n","    [theme=dark] .colab-df-convert {\n","      background-color: #3B4455;\n","      fill: #D2E3FC;\n","    }\n","\n","    [theme=dark] .colab-df-convert:hover {\n","      background-color: #434B5C;\n","      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n","      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n","      fill: #FFFFFF;\n","    }\n","  </style>\n","\n","      <script>\n","        const buttonEl =\n","          document.querySelector('#df-5b6b8314-4eb4-4c3f-b8d2-e91e119e63da button.colab-df-convert');\n","        buttonEl.style.display =\n","          google.colab.kernel.accessAllowed ? 'block' : 'none';\n","\n","        async function convertToInteractive(key) {\n","          const element = document.querySelector('#df-5b6b8314-4eb4-4c3f-b8d2-e91e119e63da');\n","          const dataTable =\n","            await google.colab.kernel.invokeFunction('convertToInteractive',\n","                                                     [key], {});\n","          if (!dataTable) return;\n","\n","          const docLinkHtml = 'Like what you see? Visit the ' +\n","            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n","            + ' to learn more about interactive tables.';\n","          element.innerHTML = '';\n","          dataTable['output_type'] = 'display_data';\n","          await google.colab.output.renderOutput(dataTable, element);\n","          const docLink = document.createElement('div');\n","          docLink.innerHTML = docLinkHtml;\n","          element.appendChild(docLink);\n","        }\n","      </script>\n","    </div>\n","  </div>\n","  "]},"metadata":{},"execution_count":8}]},{"cell_type":"markdown","metadata":{"id":"v1IrblezX6AN"},"source":["# [Part of Speech for Bengali (POS)](https://nlp.johnsnowlabs.com/2021/01/20/pos_msri_bn.html)\n","![bn](https://www.worldometers.info/img/flags/small/tn_bg-flag.gif)"]},{"cell_type":"code","metadata":{"colab":{"base_uri":"https://localhost:8080/","height":436},"id":"qs_78lxnXHsS","executionInfo":{"status":"ok","timestamp":1650025821612,"user_tz":-300,"elapsed":7031,"user":{"displayName":"ahmed lone","userId":"02458088882398909889"}},"outputId":"7faeb93b-5a90-4bf0-8994-754056ce9564"},"source":["# Bengali for : 'The village is also called 'Mod' in Tora language' \n","nlu.load(\"bn.pos\").predict(\"বাসস্থান-ঘরগৃহস্থালি তোড়া ভাষায় গ্রামকেও বলে ` মোদ ' ৷\")"],"execution_count":9,"outputs":[{"output_type":"stream","name":"stdout","text":["pos_msri download started this may take some time.\n","Approximate size to download 806.5 KB\n","[OK!]\n","sentence_detector_dl download started this may take some time.\n","Approximate size to download 354.6 KB\n","[OK!]\n"]},{"output_type":"execute_result","data":{"text/plain":["   pos                 token\n","0   NN  বাসস্থান-ঘরগৃহস্থালি\n","0  NNP                  তোড়া\n","0   NN                 ভাষায়\n","0   NN              গ্রামকেও\n","0   VM                   বলে\n","0  SYM                     `\n","0   NN                   মোদ\n","0  SYM                     '\n","0  SYM                     ৷"],"text/html":["\n","  <div id=\"df-2635da26-c8b3-4916-96c5-566b4bddb0b5\">\n","    <div class=\"colab-df-container\">\n","      <div>\n","<style scoped>\n","    .dataframe tbody tr th:only-of-type {\n","        vertical-align: middle;\n","    }\n","\n","    .dataframe tbody tr th {\n","        vertical-align: top;\n","    }\n","\n","    .dataframe thead th {\n","        text-align: right;\n","    }\n","</style>\n","<table border=\"1\" class=\"dataframe\">\n","  <thead>\n","    <tr style=\"text-align: right;\">\n","      <th></th>\n","      <th>pos</th>\n","      <th>token</th>\n","    </tr>\n","  </thead>\n","  <tbody>\n","    <tr>\n","      <th>0</th>\n","      <td>NN</td>\n","      <td>বাসস্থান-ঘরগৃহস্থালি</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>NNP</td>\n","      <td>তোড়া</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>NN</td>\n","      <td>ভাষায়</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>NN</td>\n","      <td>গ্রামকেও</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>VM</td>\n","      <td>বলে</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>SYM</td>\n","      <td>`</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>NN</td>\n","      <td>মোদ</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>SYM</td>\n","      <td>'</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>SYM</td>\n","      <td>৷</td>\n","    </tr>\n","  </tbody>\n","</table>\n","</div>\n","      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-2635da26-c8b3-4916-96c5-566b4bddb0b5')\"\n","              title=\"Convert this dataframe to an interactive table.\"\n","              style=\"display:none;\">\n","        \n","  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n","       width=\"24px\">\n","    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n","    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n","  </svg>\n","      </button>\n","      \n","  <style>\n","    .colab-df-container {\n","      display:flex;\n","      flex-wrap:wrap;\n","      gap: 12px;\n","    }\n","\n","    .colab-df-convert {\n","      background-color: #E8F0FE;\n","      border: none;\n","      border-radius: 50%;\n","      cursor: pointer;\n","      display: none;\n","      fill: #1967D2;\n","      height: 32px;\n","      padding: 0 0 0 0;\n","      width: 32px;\n","    }\n","\n","    .colab-df-convert:hover {\n","      background-color: #E2EBFA;\n","      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n","      fill: #174EA6;\n","    }\n","\n","    [theme=dark] .colab-df-convert {\n","      background-color: #3B4455;\n","      fill: #D2E3FC;\n","    }\n","\n","    [theme=dark] .colab-df-convert:hover {\n","      background-color: #434B5C;\n","      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n","      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n","      fill: #FFFFFF;\n","    }\n","  </style>\n","\n","      <script>\n","        const buttonEl =\n","          document.querySelector('#df-2635da26-c8b3-4916-96c5-566b4bddb0b5 button.colab-df-convert');\n","        buttonEl.style.display =\n","          google.colab.kernel.accessAllowed ? 'block' : 'none';\n","\n","        async function convertToInteractive(key) {\n","          const element = document.querySelector('#df-2635da26-c8b3-4916-96c5-566b4bddb0b5');\n","          const dataTable =\n","            await google.colab.kernel.invokeFunction('convertToInteractive',\n","                                                     [key], {});\n","          if (!dataTable) return;\n","\n","          const docLinkHtml = 'Like what you see? Visit the ' +\n","            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n","            + ' to learn more about interactive tables.';\n","          element.innerHTML = '';\n","          dataTable['output_type'] = 'display_data';\n","          await google.colab.output.renderOutput(dataTable, element);\n","          const docLink = document.createElement('div');\n","          docLink.innerHTML = docLinkHtml;\n","          element.appendChild(docLink);\n","        }\n","      </script>\n","    </div>\n","  </div>\n","  "]},"metadata":{},"execution_count":9}]},{"cell_type":"markdown","metadata":{"id":"RrhRJ6vFZHu1"},"source":["# [Amharic Part of Speech (POS)](https://nlp.johnsnowlabs.com/2021/01/20/pos_ud_att_am.html)\n","![am](https://www.worldometers.info/img/flags/small/tn_am-flag.gif)"]},{"cell_type":"code","metadata":{"colab":{"base_uri":"https://localhost:8080/","height":530},"id":"hrSGw3taZjBf","executionInfo":{"status":"ok","timestamp":1650025827729,"user_tz":-300,"elapsed":6153,"user":{"displayName":"ahmed lone","userId":"02458088882398909889"}},"outputId":"152ba3a1-eb45-4bda-f7b0-f1151adee160"},"source":["# Amharic for : ' \"Son, finish the job,\" he said.' \n","nlu.load('am.pos').predict('ልጅ ኡ ን ሥራ ው ን አስጨርስ ኧው ኣል ኧሁ ።\"')"],"execution_count":10,"outputs":[{"output_type":"stream","name":"stdout","text":["pos_ud_att download started this may take some time.\n","Approximate size to download 300.5 KB\n","[OK!]\n","sentence_detector_dl download started this may take some time.\n","Approximate size to download 354.6 KB\n","[OK!]\n"]},{"output_type":"execute_result","data":{"text/plain":["     pos  token\n","0   NOUN     ልጅ\n","0    DET      ኡ\n","0   PART      ን\n","0   NOUN     ሥራ\n","0    DET      ው\n","0   PART      ን\n","0   VERB  አስጨርስ\n","0   PRON     ኧው\n","0    AUX     ኣል\n","0   PRON     ኧሁ\n","0  PUNCT      ።\n","0   NOUN      \""],"text/html":["\n","  <div id=\"df-4f5e890a-e68f-4529-8c2d-bb3a9d32b902\">\n","    <div class=\"colab-df-container\">\n","      <div>\n","<style scoped>\n","    .dataframe tbody tr th:only-of-type {\n","        vertical-align: middle;\n","    }\n","\n","    .dataframe tbody tr th {\n","        vertical-align: top;\n","    }\n","\n","    .dataframe thead th {\n","        text-align: right;\n","    }\n","</style>\n","<table border=\"1\" class=\"dataframe\">\n","  <thead>\n","    <tr style=\"text-align: right;\">\n","      <th></th>\n","      <th>pos</th>\n","      <th>token</th>\n","    </tr>\n","  </thead>\n","  <tbody>\n","    <tr>\n","      <th>0</th>\n","      <td>NOUN</td>\n","      <td>ልጅ</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>DET</td>\n","      <td>ኡ</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>PART</td>\n","      <td>ን</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>NOUN</td>\n","      <td>ሥራ</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>DET</td>\n","      <td>ው</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>PART</td>\n","      <td>ን</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>VERB</td>\n","      <td>አስጨርስ</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>PRON</td>\n","      <td>ኧው</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>AUX</td>\n","      <td>ኣል</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>PRON</td>\n","      <td>ኧሁ</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>PUNCT</td>\n","      <td>።</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>NOUN</td>\n","      <td>\"</td>\n","    </tr>\n","  </tbody>\n","</table>\n","</div>\n","      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-4f5e890a-e68f-4529-8c2d-bb3a9d32b902')\"\n","              title=\"Convert this dataframe to an interactive table.\"\n","              style=\"display:none;\">\n","        \n","  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n","       width=\"24px\">\n","    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n","    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n","  </svg>\n","      </button>\n","      \n","  <style>\n","    .colab-df-container {\n","      display:flex;\n","      flex-wrap:wrap;\n","      gap: 12px;\n","    }\n","\n","    .colab-df-convert {\n","      background-color: #E8F0FE;\n","      border: none;\n","      border-radius: 50%;\n","      cursor: pointer;\n","      display: none;\n","      fill: #1967D2;\n","      height: 32px;\n","      padding: 0 0 0 0;\n","      width: 32px;\n","    }\n","\n","    .colab-df-convert:hover {\n","      background-color: #E2EBFA;\n","      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n","      fill: #174EA6;\n","    }\n","\n","    [theme=dark] .colab-df-convert {\n","      background-color: #3B4455;\n","      fill: #D2E3FC;\n","    }\n","\n","    [theme=dark] .colab-df-convert:hover {\n","      background-color: #434B5C;\n","      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n","      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n","      fill: #FFFFFF;\n","    }\n","  </style>\n","\n","      <script>\n","        const buttonEl =\n","          document.querySelector('#df-4f5e890a-e68f-4529-8c2d-bb3a9d32b902 button.colab-df-convert');\n","        buttonEl.style.display =\n","          google.colab.kernel.accessAllowed ? 'block' : 'none';\n","\n","        async function convertToInteractive(key) {\n","          const element = document.querySelector('#df-4f5e890a-e68f-4529-8c2d-bb3a9d32b902');\n","          const dataTable =\n","            await google.colab.kernel.invokeFunction('convertToInteractive',\n","                                                     [key], {});\n","          if (!dataTable) return;\n","\n","          const docLinkHtml = 'Like what you see? Visit the ' +\n","            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n","            + ' to learn more about interactive tables.';\n","          element.innerHTML = '';\n","          dataTable['output_type'] = 'display_data';\n","          await google.colab.output.renderOutput(dataTable, element);\n","          const docLink = document.createElement('div');\n","          docLink.innerHTML = docLinkHtml;\n","          element.appendChild(docLink);\n","        }\n","      </script>\n","    </div>\n","  </div>\n","  "]},"metadata":{},"execution_count":10}]},{"cell_type":"markdown","metadata":{"id":"h2iaJnZJZ4Ci"},"source":["# [Thai Sentiment Classification](https://nlp.johnsnowlabs.com/2021/01/14/sentiment_jager_use_th.html)\n","![th](https://www.worldometers.info/img/flags/small/tn_th-flag.gif)\n","\n","\n"]},{"cell_type":"code","metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"xU_xDwEpZkqs","outputId":"fa0fa113-564c-4e20-d8e1-a2a6c7e6968f"},"source":["#  thai for :  'I love peanut butter and jelly!' \n","nlu.load('th.classify.sentiment').predict('ฉันชอบเนยถั่วและเยลลี่!')[['sentiment','sentiment_confidence']]"],"execution_count":null,"outputs":[{"output_type":"stream","name":"stdout","text":["sentiment_jager_use download started this may take some time.\n","Approximate size to download 11.9 MB\n","[OK!]\n","tfhub_use_multi download started this may take some time.\n","Approximate size to download 247.6 MB\n","[OK!]\n","tfhub_use_multi download started this may take some time.\n","Approximate size to download 247.6 MB\n","[OK!]\n","tfhub_use_multi download started this may take some time.\n","Approximate size to download 247.6 MB\n","[OK!]\n","tfhub_use_multi download started this may take some time.\n","Approximate size to download 247.6 MB\n","[OK!]\n","tfhub_use_multi download started this may take some time.\n","Approximate size to download 247.6 MB\n","[OK!]\n","tfhub_use_multi download started this may take some time.\n","Approximate size to download 247.6 MB\n","[OK!]\n","tfhub_use_multi download started this may take some time.\n","Approximate size to download 247.6 MB\n","[OK!]\n","tfhub_use_multi download started this may take some time.\n","Approximate size to download 247.6 MB\n","[OK!]\n","tfhub_use_multi download started this may take some time.\n","Approximate size to download 247.6 MB\n","[OK!]\n","tfhub_use_multi download started this may take some time.\n","Approximate size to download 247.6 MB\n","[OK!]\n","tfhub_use_multi download started this may take some time.\n","Approximate size to download 247.6 MB\n","[OK!]\n","tfhub_use_multi download started this may take some time.\n","Approximate size to download 247.6 MB\n","[OK!]\n","tfhub_use_multi download started this may take some time.\n","Approximate size to download 247.6 MB\n","[OK!]\n","tfhub_use_multi download started this may take some time.\n","Approximate size to download 247.6 MB\n","[OK!]\n","tfhub_use_multi download started this may take some time.\n","Approximate size to download 247.6 MB\n","[OK!]\n","tfhub_use_multi download started this may take some time.\n","Approximate size to download 247.6 MB\n","[OK!]\n","tfhub_use_multi download started this may take some time.\n","Approximate size to download 247.6 MB\n","[OK!]\n","tfhub_use_multi download started this may take some time.\n","Approximate size to download 247.6 MB\n","[OK!]\n","tfhub_use_multi download started this may take some time.\n","Approximate size to download 247.6 MB\n","[OK!]\n","tfhub_use_multi download started this may take some time.\n","Approximate size to download 247.6 MB\n","[OK!]\n","tfhub_use_multi download started this may take some time.\n","Approximate size to download 247.6 MB\n","[OK!]\n","tfhub_use_multi download started this may take some time.\n","Approximate size to download 247.6 MB\n","[OK!]\n","tfhub_use_multi download started this may take some time.\n","Approximate size to download 247.6 MB\n","[OK!]\n","tfhub_use_multi download started this may take some time.\n","Approximate size to download 247.6 MB\n","[OK!]\n","tfhub_use_multi download started this may take some time.\n","Approximate size to download 247.6 MB\n","[OK!]\n","tfhub_use_multi download started this may take some time.\n","Approximate size to download 247.6 MB\n","[OK!]\n","tfhub_use_multi download started this may take some time.\n","Approximate size to download 247.6 MB\n","[OK!]\n","tfhub_use_multi download started this may take some time.\n","Approximate size to download 247.6 MB\n","[OK!]\n","tfhub_use_multi download started this may take some time.\n","Approximate size to download 247.6 MB\n","[OK!]\n","tfhub_use_multi download started this may take some time.\n","Approximate size to download 247.6 MB\n","[OK!]\n","tfhub_use_multi download started this may take some time.\n","Approximate size to download 247.6 MB\n","[OK!]\n","tfhub_use_multi download started this may take some time.\n","Approximate size to download 247.6 MB\n","[OK!]\n","tfhub_use_multi download started this may take some time.\n","Approximate size to download 247.6 MB\n","[OK!]\n","tfhub_use_multi download started this may take some time.\n","Approximate size to download 247.6 MB\n","[OK!]\n","tfhub_use_multi download started this may take some time.\n","Approximate size to download 247.6 MB\n","[OK!]\n","tfhub_use_multi download started this may take some time.\n","Approximate size to download 247.6 MB\n","[OK!]\n","tfhub_use_multi download started this may take some time.\n","Approximate size to download 247.6 MB\n","[OK!]\n","tfhub_use_multi download started this may take some time.\n","Approximate size to download 247.6 MB\n","[OK!]\n","tfhub_use_multi download started this may take some time.\n","Approximate size to download 247.6 MB\n","[OK!]\n","tfhub_use_multi download started this may take some time.\n","Approximate size to download 247.6 MB\n","[OK!]\n","tfhub_use_multi download started this may take some time.\n","Approximate size to download 247.6 MB\n","[OK!]\n","tfhub_use_multi download started this may take some time.\n","Approximate size to download 247.6 MB\n","[OK!]\n","tfhub_use_multi download started this may take some time.\n","Approximate size to download 247.6 MB\n","[OK!]\n","tfhub_use_multi download started this may take some time.\n","Approximate size to download 247.6 MB\n","[OK!]\n","tfhub_use_multi download started this may take some time.\n","Approximate size to download 247.6 MB\n","[OK!]\n","tfhub_use_multi download started this may take some time.\n","Approximate size to download 247.6 MB\n","[OK!]\n","tfhub_use_multi download started this may take some time.\n","Approximate size to download 247.6 MB\n","[OK!]\n","tfhub_use_multi download started this may take some time.\n","Approximate size to download 247.6 MB\n","[OK!]\n","tfhub_use_multi download started this may take some time.\n","Approximate size to download 247.6 MB\n","[OK!]\n","tfhub_use_multi download started this may take some time.\n","Approximate size to download 247.6 MB\n","[OK!]\n","tfhub_use_multi download started this may take some time.\n","Approximate size to download 247.6 MB\n","[OK!]\n","tfhub_use_multi download started this may take some time.\n","Approximate size to download 247.6 MB\n","[OK!]\n","tfhub_use_multi download started this may take some time.\n","Approximate size to download 247.6 MB\n","[OK!]\n","tfhub_use_multi download started this may take some time.\n","Approximate size to download 247.6 MB\n","[OK!]\n","tfhub_use_multi download started this may take some time.\n","Approximate size to download 247.6 MB\n","[OK!]\n","tfhub_use_multi download started this may take some time.\n","Approximate size to download 247.6 MB\n","[OK!]\n","tfhub_use_multi download started this may take some time.\n","Approximate size to download 247.6 MB\n","[OK!]\n","tfhub_use_multi download started this may take some time.\n","Approximate size to download 247.6 MB\n","[OK!]\n","tfhub_use_multi download started this may take some time.\n","Approximate size to download 247.6 MB\n","[OK!]\n","tfhub_use_multi download started this may take some time.\n","Approximate size to download 247.6 MB\n","[OK!]\n","tfhub_use_multi download started this may take some time.\n","Approximate size to download 247.6 MB\n","[OK!]\n","tfhub_use_multi download started this may take some time.\n","Approximate size to download 247.6 MB\n","[OK!]\n","tfhub_use_multi download started this may take some time.\n","Approximate size to download 247.6 MB\n","[OK!]\n","tfhub_use_multi download started this may take some time.\n","Approximate size to download 247.6 MB\n","[OK!]\n","tfhub_use_multi download started this may take some time.\n","Approximate size to download 247.6 MB\n","[OK!]\n","tfhub_use_multi download started this may take some time.\n","Approximate size to download 247.6 MB\n","[OK!]\n","tfhub_use_multi download started this may take some time.\n","Approximate size to download 247.6 MB\n","[OK!]\n","tfhub_use_multi download started this may take some time.\n","Approximate size to download 247.6 MB\n","[OK!]\n","tfhub_use_multi download started this may take some time.\n","Approximate size to download 247.6 MB\n","[OK!]\n","tfhub_use_multi download started this may take some time.\n","Approximate size to download 247.6 MB\n","[OK!]\n","tfhub_use_multi download started this may take some time.\n","Approximate size to download 247.6 MB\n","[OK!]\n","tfhub_use_multi download started this may take some time.\n","Approximate size to download 247.6 MB\n","[OK!]\n","tfhub_use_multi download started this may take some time.\n","Approximate size to download 247.6 MB\n","[OK!]\n","tfhub_use_multi download started this may take some time.\n","Approximate size to download 247.6 MB\n","[OK!]\n","tfhub_use_multi download started this may take some time.\n","Approximate size to download 247.6 MB\n","[OK!]\n","tfhub_use_multi download started this may take some time.\n","Approximate size to download 247.6 MB\n","[OK!]\n","tfhub_use_multi download started this may take some time.\n","Approximate size to download 247.6 MB\n","[ | ]"]}]},{"cell_type":"markdown","metadata":{"id":"jM7FYdeKaI8t"},"source":["# [Arabic Named Entity Recognition (NER)](https://nlp.johnsnowlabs.com/2020/12/05/aner_cc_300d_ar.html)\n","![ar](https://www.worldometers.info/img/flags/small/tn_sa-flag.gif)"]},{"cell_type":"code","metadata":{"id":"UxN70dp4Z78a"},"source":["# Arabic for : 'In 1918, the forces of the Arab Revolt liberated Damascus with the help of the British'  \n","nlu.load('ar.ner').predict('في عام 1918 حررت قوات الثورة العربية دمشق بمساعدة من الإنكليز',output_level='chunk')[['entities_confidence','ner_confidence','entities']]"],"execution_count":null,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"TMGOOL2lFGDM"},"source":["# [Part of Speech for Latin](https://nlp.johnsnowlabs.com/2020/07/29/pos_ud_llct_la.html)\n","![la](https://www.worldometers.info/img/flags/small/tn_it-flag.gif)"]},{"cell_type":"code","metadata":{"id":"-H0xCxMaD1_h"},"source":["# Latin for : 'Elias East called it Regum Aquilus, the English Medicine Writer Evans Dush in Progress Madikna Anesthesia Yat Hygen.'\n","nlu.load(\"la.pos\").predict(\"Alius est esse regem Aquilonis, et de Anglis medicus et nives Ioannes dux in progressus medicinae anesthesia et hygiene.\")"],"execution_count":null,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"v4JAGLwRGv71"},"source":["# [Latin Lemmatizer](https://nlp.johnsnowlabs.com/2020/07/29/lemma_la.html)\n","![la](https://www.worldometers.info/img/flags/small/tn_it-flag.gif)"]},{"cell_type":"code","metadata":{"id":"90fxv8NqFgxg"},"source":["# Latin for : 'Elias East called it Regum Aquilus, the English Medicine Writer Evans Dush in Progress Madikna Anesthesia Yat Hygen.'\n","nlu.load(\"lv.lemma\").predict(\"Alius est esse regem Aquilonis, et de Anglis medicus et nives Ioannes dux in progressus medicinae anesthesia et hygiene.\")"],"execution_count":null,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"Rw2ZKIy2HDM0"},"source":["# [Stop Words Cleaner for Latin](https://nlp.johnsnowlabs.com/2020/07/14/stopwords_la.html)\n","![la](https://www.worldometers.info/img/flags/small/tn_it-flag.gif)"]},{"cell_type":"code","metadata":{"id":"yNLSF8LqHCb4"},"source":["# Latin for : 'Elias East called it Regum Aquilus, the English Medicine Writer Evans Dush in Progress Madikna Anesthesia Yat Hygen.'\n","nlu.load(\"lv.pos\").predict(\"Alius est esse regem Aquilonis, et de Anglis medicus et nives Ioannes dux in progressus medicinae anesthesia et hygiene.\")"],"execution_count":null,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"vpt5z_9kHc3O"},"source":["![ru](https://www.worldometers.info/img/flags/small/tn_rs-flag.gif)\n","# [Stop Words Cleaner for Russian](https://nlp.johnsnowlabs.com/2020/07/14/stopwords_ru.html)"]},{"cell_type":"code","metadata":{"id":"FIWhNEPzHQE2"},"source":["# Russian  for : 'In addition to being the king of the north, Jon Snow is an English physician and leader in the development of anesthesia and medical hygiene.'\n","nlu.load(\"ru.stopwords\").predict(\"Помимо того, что он король севера, Джон Сноу - английский врач и лидер в разработке анестезии и медицинской гигиены.\")"],"execution_count":null,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"EhoMFWheIGzh"},"source":["# [Detect Persons, Locations, Organizations and Misc Entities in Russian (WikiNER 6B 100)](https://nlp.johnsnowlabs.com/2020/03/16/wikiner_6B_100_ru.html)\n","![ru](https://www.worldometers.info/img/flags/small/tn_rs-flag.gif)"]},{"cell_type":"code","metadata":{"id":"-kmEhzfqIGzq"},"source":["\"\"\" Russian  for : 'William Henry Gates III (born October 28, 1955) is an American business tycoon, software developer, investor, and philanthropist.\n"," He is best known as the co-founder of Microsoft Corporation. During his career at Microsoft, Gates served as chairman, chief Chief Executive Officer (CEO), President and Chief Software Architect,\n"," and was the largest individual shareholder until May 2014. He is one of the most famous entrepreneurs and pioneers of the microcomputer revolution of the 1970s and 1980s. Gates was born and \n"," raised in Seattle, Washington in 1975 with childhood friend Paul Allen in Albuquerque, New Mexico, and founded Microsoft. it has become the world's largest personal computer software company. \n"," Gates ran the company in as chairman and CEO until he stepped down as CEO in January 2000, but remained chairman and became Chief Software Architect. Late 1990s \n"," Gates was criticized for his business tactics, which were considered anti-competitive. This opinion has been confirmed by numerous court decisions. In June 2006,\n","  Gates announced that he would move to part-time at Microsoft and work full-time at the Bill and Melinda Gates Foundation, a private charitable foundation that he and\n","   his wife Melinda Gates set up in 2000. [9] Gradually he transferred his duties to Ray Ozzy and Craig Mundy. He stepped down as President of Microsoft in February 2014 and\n","    took on a new position as a technology consultant to support newly appointed CEO Satya Nadella.' \"\"\"\n","nlu.load(\"ru.ner\").predict(\"\"\"Уильям Генри Гейтс III (родился 28 октября 1955 года) - американский бизнес-магнат, разработчик программного обеспечения, инвестор и филантроп.\n"," Он наиболее известен как соучредитель корпорации Microsoft. За время своей карьеры в Microsoft Гейтс занимал должности председателя, главного исполнительного директора (CEO),\n"," президента и главного архитектора программного обеспечения, а также был крупнейшим индивидуальным акционером до мая 2014 года.\n","  Он является одним из самых известных предпринимателей и пионеров микрокомпьютерная революция 1970-х и 1980-х годов. Гейтс родился и вырос в Сиэтле, \n","  штат Вашингтон, в 1975 году вместе с другом детства Полом Алленом в Альбукерке, штат Нью-Мексико, и основал компанию Microsoft.\n","   она стала крупнейшей в мире компанией-разработчиком программного обеспечения для персональных компьютеров. Гейтс руководил компанией в качестве председателя и генерального директора,\n","    пока в январе 2000 года не ушел с поста генерального директора, но остался председателем и стал главным архитектором программного обеспечения. В конце 1990-х Гейтс подвергся критике за свою деловую тактику, \n","    которая считалась антиконкурентной. Это мнение было подтверждено многочисленными судебными решениями. В июне 2006 года Гейтс объявил,\n","     что перейдет на неполный рабочий день в Microsoft и будет работать на полную ставку в Фонде Билла и Мелинды Гейтс, частном благотворительном фонде,\n","      который он и его жена Мелинда Гейтс создали в 2000 году. [ 9] Постепенно он передал свои обязанности Рэю Оззи и Крейгу Манди.\n","       Он ушел с поста президента Microsoft в феврале 2014 года и занял новую должность консультанта по технологиям для поддержки вновь назначенного генерального директора Сатья Наделла.\"\"\")"],"execution_count":null,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"DyH8MDRIKEwO"},"source":["# [Part of Speech for Russian](https://nlp.johnsnowlabs.com/2020/03/12/pos_ud_gsd_ru.html)\n","![ru](https://www.worldometers.info/img/flags/small/tn_rs-flag.gif)"]},{"cell_type":"code","metadata":{"id":"r0xfEM9uI00r"},"source":["# Russian for : 'In addition to being the king of the north, Jon Snow is an English physician and leader in the development of anesthesia and medical hygiene.'\n","nlu.load(\"ru.pos\").predict(\"Помимо того, что он король севера, Джон Сноу - английский врач и лидер в разработке анестезии и медицинской гигиены.\")"],"execution_count":null,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"P4vqpJFXKTlk"},"source":["# [Russian Lemmatizer](https://nlp.johnsnowlabs.com/2020/03/12/lemma_ru.html)\n","![ru](https://www.worldometers.info/img/flags/small/tn_rs-flag.gif)"]},{"cell_type":"code","metadata":{"id":"wWsne8wfKDBx"},"source":["# Russian for : 'In addition to being the king of the north, Jon Snow is an English physician and leader in the development of anesthesia and medical hygiene.'\n","nlu.load(\"ru.lemma\").predict(\"Помимо того, что он король севера, Джон Сноу - английский врач и лидер в разработке анестезии и медицинской гигиены.\")"],"execution_count":null,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"tUDsioMYM3V_"},"source":["# [Part of Speech for Arabic](https://nlp.johnsnowlabs.com/2020/11/30/pos_ud_padt_ar.html)\n","![ar](https://www.worldometers.info/img/flags/small/tn_sa-flag.gif)"]},{"cell_type":"code","metadata":{"id":"3A8KYw8NKoYI"},"source":["# Arabic for : 'Cristiano Ronaldo is a Portuguese professional player who plays for the Portugal national football team'\n","nlu.load(\"ar.pos\").predict(\"كرستيانو رونالدو لاعب برتغالي محترف يلعب في صفوف منتخب البرتغال لكرة القدم\")"],"execution_count":null,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"bY-lTI-1Nab1"},"source":["# [Arabic Lemmatizer](https://nlp.johnsnowlabs.com/2020/11/28/lemma_ar.html)\n","![ar](https://www.worldometers.info/img/flags/small/tn_sa-flag.gif)"]},{"cell_type":"code","metadata":{"id":"LO4gw_VgNKVD"},"source":["# Arabic for : 'He has completed all his tasks in time therefore he is allowed to go and play for a while .'\n","nlu.load(\"ar.lemma\").predict(\"أنهى جميع مهامه في الوقت المحدد لذلك سمح له بالذهاب واللعب لفترة.ا\")"],"execution_count":null,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"1Mk_9ZrCOHWO"},"source":["# [Stop Words Cleaner for Arabic](https://nlp.johnsnowlabs.com/2020/07/14/stopwords_ar.html)\n","![ar](https://www.worldometers.info/img/flags/small/tn_sa-flag.gif)"]},{"cell_type":"code","metadata":{"id":"cNTJbq-BOIdO"},"source":["# Arabic for : 'Other than being the King of the North, Jon Snow is an English physician and pioneer in the development of anesthesia and hygiene.'\n","nlu.load(\"ar.stopwords\").predict(\"بخلاف كونه ملك الشمال ، فإن جون سنو طبيب إنجليزي ورائد في تطوير التخدير والنظافة الطبية.\")"],"execution_count":null,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"foSWDhkN-uN6"},"source":["# There are many more models!\n","## Checkout [the Modelshub](https://nlp.johnsnowlabs.com/models) and the [NLU Namespace](https://nlu.johnsnowlabs.com/docs/en/spellbook) for more models"]},{"cell_type":"code","metadata":{"id":"CFsZ8qyE-4i5"},"source":[""],"execution_count":null,"outputs":[]}]}